A meta-analysis delivered to the offices of JAMA
I never meta analysis I liked
I have a love-hate relationship with meta-analyses (here after called metas). I have never understood why metas are always put on the top of the evidence pyramid as they are only as good, or as bad, as the studies that go into the analysis. Metas don’t belongs on the evidence pyramid any more than spam hash belongs on the food pyramid, but that’s me.
Metas do have their uses. As a specialist, I like metas in my field as they give a nice overview of a topic and offer the opportunity to see if my understanding of a topic matches the metas. I usually know most, but never all, the primary literature and metas allow me fill in gaps in my knowledge and, if I was in disagreement with the meta, to assess my understanding.
For medicine outside of infectious diseases metas give me an understanding of current thinking of a literature I will never read.
For pseudo-medicines? The theory behind pseudo-medical metas is that if you take all cow pies and place them in one big pile, you get gold. The reality is you get a huge cow pie. For metas of homeopathy or acupuncture, it is usually garbage in garbage out; meta-analysis of fiction can only lead to fictional results. Acupuncture metas are particularly annoying as it is rare that any of the compared interventions are the same: different acupuncture styles (ear or Chinese or electro etc), different points used, different frequency and duration of treatment. And most pseudo-medical metas have the same conclusion: maybe some effect, but poor quality studies, so more studies need be done.
A meta for SMT?
But what about spinal manipulation therapy (SMT) for low back pain? There is no prior plausibility that suggests SMT would not have effects on low back pain. We are apes, and apes find hands-on care beneficial. And SMT is not necessarily synonymous with chiropractic, although chiropractors are the main purveyors of SMT. So while there is the urge to judge a therapy by the company it keeps, it may be that SMT has efficacy independent of the pseudo-science of innate intelligence, subluxations, and chiropractic manipulations.
JAMA had a meta-analysis of SMT for low back pain suggesting to a superficial reading that SMT was minimally beneficial: “Association of Spinal Manipulative Therapy With Clinical Benefit and Harm for Acute Low Back Pain: Systematic Review and Meta-analysis.”
I would like to know the primary literature that went into the construction of this meta. Metas are not laws or sausage; you really need to know what they are made of. Is it made of high quality studies? Or cow pie?
There are the usual summary tables, one for pain:
And one for short term function. Let’s go with the pain references, shall we, and see what we can discover.
There were two references in the meta-analysis that were “Comparison group, sham” and both were considered to favor manipulation. There were only two sham controlled trials in the whole meta. And if the studies are not sham/placebo controlled they are likely useless.
This was the best study ?!?
There was “Assessment of diclofenac or spinal manipulative therapy, or both, in addition to recommended first-line treatment for acute low back pain: a randomised controlled trial.” In this study the sham was a detuned pulse ultrasound applied to the patient as long as the manipulation. Not really a manipulation sham controlled trial. And the SMT? We have no idea what the patients received:
Consistent with contemporary best clinical practice, the physiotherapist will adjust the treatment to the clinical presentation of the patient rather than apply the same treatment to all patients.
These physiotherapists used:
the application of manually applied postero-anteriorly directed forces to all levels of the lumbar spine.
Whatever that might be. And the results?
Neither diclofenac nor spinal manipulative therapy had a statistically significant effect on the secondary outcomes of pain, disability, function, or global perceived effect at any time point.
It is odd that an essentially negative original study is analyzed as being suggestive of SMT efficacy in the meta-analysis. I can see no explanation for this except the magic of metas.
And this was considered to be the best study in the group with a rating of 8. Completely negative. And they say:
high-quality studies tended to report larger benefits.
Hmmm. The highest rated had no benefit I can find.
Failed blinding with extra pseudo-science
The other sham study, rated a mere 3, was “A randomized clinical trial comparing chiropractic adjustments to muscle relaxants for subacute low back pain.”
In this trial they compared:
chiropractic adjustments with placebo medicine, muscle relaxants with sham adjustments, or placebo medicine with sham adjustments.
In this study patients received cervical, lumbar and/or pelvic adjustment by the DC’s best judgement (snort). In the cervical spine they used an activator for an extra bit of worthless pseudo-medicine. A completely different intervention than the first, best, study.
So again, we really don’t know what SMT the patients received and if it was comparable to the first trial. Maybe all the benefit was from the activator, he says in a voice filled with sarcasm.
The sham was a laying on of the hands with light pressure but no thrust. Obviously not a chiropractic adjustment since it would not have snap crackle pop of a ‘real’ adjustment. As a result blinding failed big time, 87% in the chiropractic group thought they received the real deal, 40% in the medical group and 20% in the control.
When you look at the graphs and error bars, even the small benefit from the unsuccessfully blinded chiropractic overlaps with the comparators.
This is a negative study, and rating it a three is overly optimistic. Still, the meta says it favors spinal manipulation.
Sham studies of the other kind of sham
The rest of the studies were “Comparison group, all other therapies”.
What were the other therapies? Let’s go through them best to worst, which is more than the editors and writers did as the footnotes are off. The table has Grunnesjö et al, as reference 35, but it is reference 37. Oops. Hyperlinks and footnotes are tough. And I know I am setting myself up for a fall here.
Reference 37 would be “A randomized controlled clinical trial of stay-active care versus manual therapy in addition to stay-active care: functional variables and pain.”
SMT compared to standard treatment. A worthless methodology. As Dr. Edzard Ernst notes:
We have been over this so many times… that I am almost a little embarrassed to explain it again: it is fairly easy to design an RCT such that it can only produce a positive result. The currently most popular way to achieve this aim in alternative medicine research is to do a ‘A B versus B’ study, where A = the experimental treatment, and B = routine care…
As I stated above, it is fairly easy to design an RCT such that it can only produce a positive result. Yet, it is arguably also unethical, perhaps even fraudulent, to do this. In my view, such RCTs amount to pseudoscience and scientific misconduct.
In this study what was the SMT? Similar to the others? Of course not. Cochrane forbid they are comparing equivalent interventions. In Grunnesjö et al they used some sort of:
Swedish manual therapy with “locking techniques”, mobilization for pelvic dysfunctions according to Kubis, with the addition of an Evjent and Hamberg locking technique and a strictly applied MET procedure.
Whatever the hell that is. Kubis appears to be the craniosacral therapy of the sacroiliac joint. What Evjent and Hamberg are doing is beyond my google skills; some sort of muscle stretching. And MET (Muscular Energy Technique) is where the patient rather than the provider, causes the muscle to contract. Also:
All patients were treated with specific mobilization or lumbar thrust techniques (manipulation) or both by the 2 physicians. In addition, the physiotherapists treated 67% of the experimental patients with specific mobilization or manipulation. According to the factorial study design, steroid injections, in case of specific findings, were allowed in half of the patients. Steroids were often given in combination with “needling” and local anesthetics. After parasacrococcygeal injections, the soft tissues were also stretched per rectum.
Steroids. Local anesthetics. Needling. Soft tissue stretching per rectum. Treatment or abuse?
They decreased significancy over time in both groups but there were not significant differences between groups.
After aggressive ‘p-hacking‘ (there were dozens of variables evaluated) they did come up with a barely significant difference: a faster decrease in pain at the last week of intervention.
This was another negative study. It was rated a 7! And again favored spinal manipulation?!? Given all those interventions that is concluded how? You and I both know it was the soft tissue stretching per rectum and not the spinal manipulation.
I am only three into the original papers. If you thought knowing what went into the production of hot dogs makes one queasy, just imagine the offal we will find in the next study.
Hot dogs have better ingredients
Bloomberg, the next 7-rated study, had five references. The first reference (32) in the table is a study that compares 101 patients:
One group was given standardized conventional and optimised activating treatment by primary health care teams. The other group received, according to a pragmatic approach, another treatment programme including manipulation, specific mobilisation, muscle stretching, autotraction and cortisone injections.
You know, the kind of methodology that will give a positive result. But given the hodgepodge of interventions in the experimental group, I think it would be impossible to credit the guaranteed positive results as “favors spinal manipulation”.
The second reference is 34. Again it points to the wrong footnote, I suspect they mean 36, but the study is more of the same, but with fewer patients, 53 instead of 101. But why it is referenced in the table since as best I can tell it was not included in the calculations. The final three references are all the same 101 patients. Four papers from one horrible study. Not bad. But my library doesn’t have access to all the Bloomberg papers so further evaluation is not forthcoming.
Pragmatic hodgepodge
The next? “Adding chiropractic manipulative therapy to standard medical care for patients with acute low back pain: results of a pragmatic randomized comparative effectiveness study.” Definition of Pragmatic: A study methodology for researchers who want their bias confirmed rather than to understand.
And the SMT intervention this time (emphasis added)?
Participants in the group receiving CMT in addition to SMC were scheduled for up to 2 visits weekly with a doctor of chiropractic (DC) for a period of 4 weeks. The initial visit with the DC included a focused history and physical examination and diagnostic imaging as indicated. Treatments consisted of HVLA manipulation as the primary approach in all cases, with ancillary treatments at the doctor’s discretion, including brief massage, the use of ice or heat in the lumbar area, stretching exercises, McKenzie exercises, advice on activities of daily living, postural/ergonomic advice; and mobilization. HVLA manipulation involves a single load or impulse “thrust” to body tissues. Patients were placed in a lateral recumbent or side-lying position with the superior or free hip and knee flexed and adducted across the midline. The chiropractor stabilized the patient’s free leg with his own leg while holding the patient’s superior shoulder. The manipulative load was applied by using a pisiform contact on the patient’s lumbar spine or sacroiliac joint while preventing motion of the patient through stabilizing holds on the shoulder and hip. The single impulse load, or thrust, was delivered by a quick, short controlled movement of the shoulder, arm, and hand combined with a slight body drop.
This 7 (?!?!?!) rated study with a hodgepodge of interventions, again different from the prior studies, ‘favors spinal manipulation?!?!?!? Seriously? This is just getting sad.
The last 7-rated study was “A randomized, controlled trial of osteopathic manipulative treatment for acute low back pain in active duty military personnel.” This time it as osteopathic manipulative treatment added to usual care. See a pattern here? And this was considered a good study by the meta, not pseudoscience and scientific misconduct. I lean towards the latter.
They used a hodgepodge of techniques (see the link) that differed, again, from the other interventions.
The next, rated a 6, “A comparison of physical therapy, chiropractic manipulation, and provision of an educational booklet for the treatment of patients with low back pain“:
For patients with low back pain, the McKenzie method of physical therapy and chiropractic manipulation had similar effects and costs, and patients receiving these treatments had only marginally better outcomes than those receiving the minimal intervention of an educational booklet.
What did they do for this marginal benefit?
a short-lever, high-velocity thrust directed specifically at a “manipulable lesion.” This procedure is typically performed with the patient lying on his or her side on a segmental table. No other physical treatments were permitted. Chiropractors evaluated patients according to their usual procedures and were allowed to make the same recommendations about exercise and activity restrictions that they usually did.
Again, more than just lousy methodology, there was more than manual therapy done. Yet, somehow, it too ‘favors spinal manipulation’ as one would expect.
Threes
The last 3’s were:
“Manipulation in the Treatment of Acute Low Back Pain “, another A vs. A + B study using orthopedic manipulation; I can’t get the original to see just what orthopedic manipulation was and how if differed from other interventions.
And:
“Acute low back pain. Comparison of two conservative treatment approaches.” They looked at:
The conservative treatments compared were passive mobilisation and manipulation of the lumbar spine and a regimen of microwave diathermy, isometric abdominal exercises and ergonomic instructions.
Another hodgepodge of interventions compared. Studies that ‘favor spinal manipulation.’ Eye roll.
The wurst
Finally the wurst, rated a 2, was “One-year follow-up comparison of the cost and effectiveness of chiropractic and physiotherapy as primary management for back pain. Subgroup analysis, recurrence, and additional health care utilization” showed no difference compared to physiotherapy although:
More patients in the chiropractic group (59%) than in the physiotherapy group (41%) sought additional health care
Favor manipulation? Of course. But it is not the spine that is being manipulated in this meta.
Conclusion: I’d rather watch sausage being made
What went into this meta-analysis?
A hodgepodge of different interventions from either negative or useless studies. In other words cow pie.
And that is what came out.
Cow pie.
Their conclusion?
Among patients with acute low back pain, spinal manipulation therapy was associated with modest improvements in pain and function at up to 6 weeks, with transient minor musculoskeletal harms.
Nonsense. Drawing any conclusions about any form of spinal manipulation for low back pain is impossible given how awful the primary literature is.
My conclusion from this Frankenstein’s Monster of a meta, where more than the brain was abby normal, a meta-analysis constructed from nonfunctioning, putrid parts?
Does SMT help for low back pain? Who knows. I have zero idea. Not that I could tell from the primary literature. Any effect of SMT is impossible to dissect out from all the other interventions in the studies. What little beneficial effect that is reported is small and likely from placebo and/or bias. And with a hodgepodge of interventions, it would appear to be a nonspecific effect, like one ape grooming another.
Do I even know what SMT is so it could be applied? Nope. I just know I do not want my muscles stretched per rectum based on this meta.
Did the lightening bolt of the meta-analytic methodolatry transform negative and/or useless studies into life? Nope.
Will this study be used to promote chiropractic SMT with all the additional pseudo-science that makes up chiropractic, setting the monster loose on the village? You bet.
The accompanying editorial, by a Professor of Evidence Based-Medicine, is the usual cautious, namby-pamby analysis that shows how EBM fails when applied to pseudo-science. A little bit of the appeal to popularity here, bit on the issues with the standard treatments there, with downplaying of the risks and underlying fiction of chiropractic. And nothing that suggests a critical review of the sausages that made up this monster. And no, I am not mixing metaphors, beer with me here.
The editorial also suggests that SMT is equal to NSAIDs. I would note that a recent BMJ meta suggests NSAIDs do not work, as noted in the confused conclusions:
NSAIDs are effective for spinal pain, but the magnitude of the difference in outcomes between the intervention and placebo groups is not clinically important.
Effective but no different than placebo? You would think this was an acupuncture meta.
And the Danish guidelines have advised against NSAIDs for the treatment acute low back pain. So yeah, I agree. SMT is likely as beneficial as NSAIDs.
I suspect the author does not spend much time in the world of pseudo-medicine as it contains the following howler:
Chiropractic therapy is not widely accepted by some traditional health care practitioners. This may be, at least in part, because some early practitioners of chiropractic care rejected the germ theory, immunizations, and other scientific advances.
Dude. That defines chiropractic today every bit as much as then. Or maybe he is referring to those who start work at 6 am.
And that is the problem with SMT. Even if SMT was effective low back pain, you would likely send your patient off to a chiropractor, with all the additional pseudo-science that entails. It would be like finding your diabetic patient is iron deficient and suggesting they replete it by doing all their dining at McDonalds.
Hardly worth it.